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A  Summary  Table  of  Intended  use  of  AWW  Related  Reports  Provided  by 
DRDC 

The  following  table  is  an  outline  of  how  the  CSEC  intends  to  use  the  reports  provided  by  DRDC  for  the  AWW  Visual  Decision 
Support  Storyboard  Project.  Listed  is  the  name,  author,  and  length  of  each  report.  Following  is  the  “Intended  Use”,  which 
summarizes  how  the  content  in  certain  sections  of  the  reports  relates  to  the  project.  The  next  three  columns  estimate  the  “weight  of 
effort’  necessary  to  utilize  the  report  section,  which  includes  the  #  of  pages  of  the  section,  the  type  of  action  (ranging  from  use  as  a 
reference,  to  review  of  all  pages),  and  the  level  of  transformation  required  from  the  given  analysis  technique  to  ACWA.  A  full 
reference  for  each  report  can  be  seen  in  Section  B  of  this  document. 


Summary  of  ACWA's  Intended  Use  of  AWW  Related  Reports  provided  by  DRDC 

Weight  of  Effort 

Report  Name 

Author 

ACWA’s  Intended  Use  of 
Report 

#  of 

Relevant 

Pages 

Action 

Transformation 

Required 

Task  Analysis  of  the  HALIFAX 
Class  Operations  Room  Officer 
(ORO),  Sensor  Weapons 
Controller  (SWC),  and  Assistant 
Sensor  Weapons  Controller 
(ASWC)  Positions:  Mission, 
Function,  and  Task 
(cont.) 

CMC 

Electronics 

Inc. 

557  Pages 
-  PDF 

Mission  Analysis  (summary  of 
domain)  to  be  used  as  reference  to 
information  about  domain. 

45  -  text 

Reference 
as  needed 

Average 

Function  Flow  Diagrams  -  First 

Level  and  Second  Level  Functions 
will  be  used  as  a  reference  to  fill 
gaps  in  FAN 

17  - 

diagram 

Reference 
as  needed 

Heavy 
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Summary  of  ACWA's  Intended  Use  of  AW) 

N  Related  Reports  provided  by  DRDC 

Weight  of  Effort 

Report  Name 

Author 

Report 

Length 

ACWA’s  lnt< 
Re 

snded  Use  of 
port 

Action 

T  ransformation 
Reguired 

Analysis  Report 

Gross  and  Critical  Task  Analysis 
will  be  reviewed  for  "tasks"  that 
require  high  cognitive  work  (ie. 
high  Decisional  Components)  and 
for  "tasks"  that  have  a  suggested 
Risk  Mitigation  of  "Decision 

Support  Aid." 

300  -  table 

Review 
table  for 
“tasks”  that 
require  high 
cognitive 
work 

Average 

Mission  Composite  Scenario  -  may 
give  ideas  for  visualization 
storyboard  prototype 

30  -  text 

Review  for 
storyboard 
scenario 
ideas 

Minimal 

Development  of 
Blueprint/Functional  Model  for 
Studying  the  Combat  System 
Functional  Requirements  of 
Canadian  Warships 

Lockheed 

Martin 

Canada  Inc. 

1956 

Pages  - 
PDF 

Functions  and  Inter-Function 
Messages  to  be  used  as  last  resort 
as  resource  to  fill  gaps  in  Cognitive 
Work  Requirements  and 

Information  Relationship 
Requirements  in  the  ACWA  model 

265  -  text, 
plus  others 
to  gain 
context  if 
relevant 

Reference 
as  needed 

Very  Heavy 

Functional  Analysis  of  the 
Canadian  Naval  Task  Group 
Operational  Planning  Process 

HumanSysfe 
ms  Inc. 

1 1 2  Pages 
-  PDF 

The  Comparison  of  OPP  to  the 
OROs'  Mental  Models  will  be  used 
as  a  reference  when  gaps  in  IRRs 
are  found  in  ACWA  modeling  and 
to  identify  Representation  Design 
Requirements  during  ACWA 
design. 

5  -  table 

Reference 
as  needed 

Heavy 
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Summary  of  ACWA's  Intended  Use  of  AWW  Related  Reports  provided  bv  DRDC 

Weight  of  Effort 

Report  Name 

Author 

Report 

Length 

ACWA’s  Intended  Use  of 
Report 

#  of 

Relevant 

Pages 

Action 

Transformation 

Reguired 

(cont.) 

Expert/Novice  Issues  were 
identified  during  the  Task  Analysis. 
All  of  these  issues  will  be  reviewed 
for  their  impact  on  ACWA  modeling 

1 8  -  table 

Review 
table  for 
Expert/ 
Novice 

Issue 

insights 

Heavy 

Actions  Requirements  and 
Information  Requirements  from  the 
Task  Analysis  will  be  used  as  a 
reference  when  gaps  in  Cognitive 
Work  Requirements  and 

Information  Relationship 
Requirements  are  found  in  ACWA 
modeling. 

1 8  -  table 

Reference 
as  needed 

Heavy 

Task  Analysis  of  the  HALIFAX 
Class  Operations  Room  Officer 
(ORO),  Sensor  Weapons 
Controller  (SWC),  and  Assistant 
Sensor  Weapons  Controller 
(ASWC)  Positions:  Information 
Flow  and  Processing  Analysis 
Report 

CMC 

Electronics 

Inc. 

59  Pages  - 
PDF 

Decisions  Required  that  have  been 
identified  by  CMC  for  each  task  in 
the  analysis  will  be  reviwed  for 
possible  insights  to  ACWA 

1 5  -  table 

Review 
table  for 

CWR 

insights 

Average 

Summary  of  Information 
Requirements  were  identified  by 
CMC  for  each  task.  This  section 
will  be  reviewed  for  possible 
insights  to  Information  Relationship 
Requirements  in  ACWA. 

1 5  -  table 

. 

Review 
table  for  IRR 
insights 

Average 
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Summary  of  ACWA's  Intended  Use  of  AW\ 

V  Related  Reports  provided  by  DRDC 

Weight  of  Effort 

Report  Name 

Author 

ACWA’s  lnt< 
Re 

jnded  Use  of 
port 

Action 

Transformation 

Required 

Recommended  Improvements 
to  the  HALIFAX  Class  Frigate 

HumanSysfe 
ms  Inc. 

42  Pages  - 
PDF 

All  recommendations  by  HSI  will 
be  reviewed  for  their  relevance  to 
the  decision  support  system 
storyboard  prototype  and 
suggested  operations  room  layout 
to  be  developed  during  this  project. 

30  -  text 

Review  all 
pages  for 
relevance  to 
tasks  of 
project. 

Minimal 

Assessing  the  Impact  of  Multi- 
Sensor  Data  Fusion  on 

Command  and  Control 

Operations  in  the  HALIFAX 

Class  Frigate: 

Recommendations  for 

Measures  of  Performance  and 
Detailed  Test  Plan 

HumanSysfe 
ms  Inc. 

67  Pages  - 
PDF  “ 

All  identified  Mea 
Performance  will 
their  insights  to  C 
and  Information 
Requirements  ini 

sures  of 
be  reviewed  for 
lognitive  Work 
Relationship 
lerit  in  the  domain 

4  -  table 

Review 
table  for 
insights  to 
CWRs  and 
IRRs  and 
cross  check 
with  current 
ACWA 
model 

Average 

Testing  Plan  and 
Section  will  be  re 
Decision-Testing 
possible  insights 

Data  Evaluation 
viewed  by  the 
Expert  for 

30  -  text 

Review  by 
DCT  expert 
for  insights 
to  DCT 

Unknown 

Review  of  the  TADMUS 

Decision  Support  System 

HumanSysfe 
ms  Inc. 

90  Pages  - 
PDF 

Included  table  of 
from  previous  an 
position  has  bee 
report  and  will  be 
relevance  to  AC\ 

ORO  "Goals" 
alysis  of  ORO 
i  included  in  this 
reviewed  for 

VA  modeling 

1  -  table 

Review 
table  for 
insights  to 
ACWA 
modeling 

Average 

25  January  2006 


Page7 


SMA  -  A  ManTech  Group 


Above  Water  Warfare  Command  and  Control  Visual  Decision  Support  Storyboard  Prototype 
W7701-4-2124/001/QCA 


Summary  of  ACWA's  Intended  Use  of  AWW  Related  Reports  provided  bv  DRDC 

Weight  of  Effort 

Report  Name 

Author 

Report 

Length 

ACWA’s  Intended  Use  of 
Report 

#  of 

Relevant 

Pages 

Action 

Transformation 

Reguired 

(cont.) 

Included  table  of  ORO  Mental 

Model  of  ORO  tasks  from  previous 
analysis  of  ORO  position  has  been 
included  in  this  report  -  information 
relevant  as  resource  -  also 
included  in  Functional  Analysis  of 
Canadian  Naval  Task  Group 
Operational  Planning  Process 
Report 

7  -  table 

content  also 
included  in 
another 
report  -  may 
be  used  as 
reference  in 
either  report 

Heavy 

Studies  of  the  U.S.  Navy  Air 
Defense  Threat  Assessment: 
Cues,  Information  Order,  and 
Impact  of  Conflicting  Data 

Pacific 

Science  and 
Engineering 
Group,  Inc. 

75  Pages 
PDF 

ACWA  could  use  the  data  list 
within  the  report  to  initiate  guided 

KE  in  an  attempt  to  identify 
Information  Relationship 
Requirements  needed  for  Decision 
Making. 

4 

Reference 
as  Needed 

Heavy 
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C  Review  of  CMC  Electronics  Inc.’s  Mission, 
Function,  Task  Analysis  (MFTA) 

C.1  Overview  of  the  MFTA  Framework 

CMC  Electronics  Inc.'  Mission  Function  Task  Analysis  is  aimed  at  the  tasks  performed 
by  the  SWC/ASWC  and  ORO  positions  on  the  Halifax  Class  Frigate.  The  framework  for 
analysis  begins  with  the  “high  level”  missions  of  the  actors.  Each  mission  is  then  de- 
aggregated  hierarchically  to  the  “tasks.”  The  hierarchy  built  by  CMC  happens  to  be  5 
levels,  however,  is  not  to  be  confused  with  Rassmussen’s  5-level  hierarchy. 

Below  is  an  example  of  the  block-diagram  de-aggregation  hierarchy  produced  by  CMC. 
The  illustration  below  is  meant  to  give  an  idea  of  the  structure  of  CMC’s  Mission 
Function  Task  Analysis  framework.  Each  block  is  referred  to  by  CMC  as  a  “Function.” 
The  “Functions”  actually  begin  as  “missions”  (CMC's  Top  Level  Functions)  and  end  with 
“tasks”  (CMC’s  Fourth  Level  Functions).  The  Top  Level,  First  Level,  and  Second  Level 
Functions  are  shown  below  in  Figure  1  for  only  a  portion  of  the  analysis. 

Figure  2  illustrates  the  format  of  the  Gross  Task  Analysis  portion  of  CMC’s  MFTA. 

CMC  does  a  more  robust  analysis  on  the  Fourth  Level  Functions  or  “tasks.”  The 
Decision  Requirements  section  of  the  Task  Analysis  is  annotated  and  will  be  considered 
for  Tasks  with  a  high  Decisional  Component  (also  annotated).  Other  artifacts  of  CMC’s 
MFTA  exist,  and  can  bee  seen  in  Appendix  A.  A  complete  list  of  the  CMC  analysis 
artifacts  are  listed,  along  with  examples,  descriptions,  and  relationships  to  ACWA  for 


Figure  1:  Example  -  CMC’s  Hierarchical  Mission  to  Task  De-aggregation  Diagram 
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CMC  ELECTRONICS  INC. 

HUMAN  FACTORS  ENGINEERING 


ANNEX  F  -  SWC/ASWC  GROSS  TAS>K  ANALYSIS  RESULTS 


DOC  NO 
1000-1242 


Top  Level: 

i 

Conduct  Transit 

First  Level: 

u 

Proceed  On  Watch 

Second  Level: 

1.1.1 

Prepare  To  Assume  Watch 

Third  Level: 

1. 1.1.1 

SWC  Task 

Task  N«  tuber: 

Source  Task: 

1.1.1.1(d) 

1.1.1.1(d) 

Discuss  watch  conduct  with  ORO 

Task  Completion  Tines 

Most  Likely:  120 
tiax:  300 

Mia :  20 

Description:  Prior  to  assuming  the  watch  the  operator  discusses  expectations  of  the  upcoming  watch  with  the  oncoming  ORO.  Discussion  points  include,  but  are  not  limited  to,  team  assignments,  and 
preparation  for  serials,  events  and  taskings  anticipated  to  occur  during  the  upcoming  watch.  The  operator  may  take  written  notes  for  future  reference. 


Initialing  Conditions: 
I .  Coming  on  watch 


Action  Requirements: 

1 .  Conduct  face-to-face  conversation 


Feedback  Required: 
l.  Verbal  Acknowledgement 


2.  Conducted  periodically  as  required 


2.  Write 


2. 


3. 


Information  Required: 

1  •  Details  of  on-going  and/or  planned  activities 

2.  Mission  requirements  and  objectives 

3.  ORO  VCommand’s  directions/intentions 


Decision  Component 

'TKffVBsfcfa^-o-dtsrisninaJ  component  that  forms  i 
moderate  amount  of  the  cognitive  content. 


3. 


Information  Available 

t .  Required  information  is  available  in  the  form  of  hard  copy 
SOPsfteferencealdocuments 

2.  Required  information/direction  is  provided  by  : 
ORO/Command 

3.  Operator  may  be  required  to  solicit  additional  information 
from  other  sbips/units 


Communication  Required: 

1 .  Face-to-Face  (Verbal) 

2. 

Frequency:  Occasionally 

Accuracy  Requrcd:  Low 

Interaction  with  ORO  1^1 


For  “tasks”  with  a  high  Decision  Component 
the  Decision  Requirements  will  provide  many 
insights  for  ACWA 


Task  Criticality  Rating  (TCR):  7  Adjusted  TCR:  7  Criticality  Rationale: 

This  task  has  high  potential  to  adversely  affect  Mission  Effectiveness/Completion  if  improperly  performed.  The  demands 
of  this  task  do  not  approach  human  performance  limits.  Further  analysis  of  this  task  is  unlikely  to  result  in  improvement 
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C.2  Places  of  Potential  Input  from  CMC’s  MFTA  to  ACWA 

The  following  diagram  shows  the  places  of  potential  input  CMC’s  Mission  Function  Task 
Analysis  of  the  SWC/ASWC  and  ORO  positions  of  the  HALIFAX  class  Frigate  may  have  for 
ACWA.  Solid  arrows  depict  artifacts  that  are  similar  in  content  to  those  of  ACWA.  Dotted 
arrows  depict  artifacts  that  have  potential  to  give  insights  to  ACWA  modeling.  The  highlighted 
artifacts,  although  the  content  they  possess  does  not  match  that  of  ACWA,  they  define  areas  of 
CMC’s  analysis  that  may  be  of  particular  interest  to  ACWA  modelers. 


10729/2004 


Places  of  Potential  Input 
from  CMC’s  MFTA  to  ACWA 

ACWA  CMC’s  MFTA  r 


o 

c 

3 


Figure  3:  Places  of  Potential  input  from  CMC’s  CTA  to  ACWA  Diagram 
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C.3 


Summary  of  Insights  from  CMC’s  MFTA  into  the  AWW 


ro 


r  ■  ujcv/i 


From  the  diagram  above,  it  can  be  seen  that  there  are  many  opportunities  for  CMC’s  analysis  to 
provide  insights  to  ACWA.  CMC's  CTA  is  focused  on  the  mission  of  an  actor  (in  this  case  the 
SWC/ASWC  and  the  ORO)  which  provides  similar  information  as  many  KE  sources  used  by 
ACWA.  The  following  are  places  where  most  insights  are  expected  to  be  found: 


Mission  Analysis  -  a  summary  of  the  domain  -  may  provide  insights  to  any  level 
of  ACWA;  however,  most  insights  from  this  section  are  expected  to  be  high 
level  goals  of  the  domain.  The  Mission  Analysis  section  will  be  used  as  a 
reference  to  information  about  the  domain,  mostly  for  ship  navigation, 
combat  suite,  sensor  management,  and  “physical  system”  knowledge,  as  those 
are  high  level  areas  to  be  included  in  the  Functional  Abstraction  Network.  This 
section  is  45  pages  long,  and  well  labeled.  Searching  for  needed 
information  should  be  fairly  straight  forward. 


CMC  provides  a  “task”  oriented  de-aggregation  of  an  actor’s  missions.  If  used 
as  a  KE  source,  the  second  and  third  levels  of  this  hierarchy  (CMC’s  First  and 
Second  Level  Functions )  may  provide  insights  to  ACWA’s  “Goal  Process 
Nodes”  (both  Goal  sub-nodes  and  Process  sub-nodes)  or  “Cognitive  Work.” 
The  First  and  Second  Level  Functions  of  CMC’s  analysis  will  be  used  as  a 
reference  if  gaps  in  the  ACWA  modeling  are  found.  Seventeen  pages  of 
Function  Flow  Diagrams  are  provided  by  CMC.  The  use  of  these  diagrams 
will  require  a  heavy  transformation  from  CMC’s  representation  as  a 
mission  to  task  de-aggregation  to  ACWA’s  goal-means  representation. 


The  lowest  level  of  de-aggregation  is  made  of  the  Fourth  Level  Functions, 
which  are  the  “tasks”  associated  with  completing  the  mission.  CMC  provides  a 
detailed  analysis  of  each  task.  The  relevant  information  for  ACWA  includes 
the  following: 

■  Decision  Component  rating,  which  rates  the  amount  of  cognitive  work 
associated  with  the  specific  “task.”  When  this  rating  is  “high”  the 
decision  “task”  will  provide  insights  to  ACWA’s  “Cognitive  Work 
Requirements.” 

■  An  explanation  of  this  decision,  labeled  as  the  Task  Decision 
Requirement  by  CMC. 

■  A  Risk  Mitigation  suggestion  is  also  provided  for  a  limited  number  of 
“tasks.”  Some  “tasks”  are  labeled  with  a  suggestion  to  provide  a 
“Decision  Support  Aid.” 

Each  “task”  will  be  reviewed  to  identify  those  with  a  high  Decision 
Component  Rating  and  a  Risk  Mitigation  suggestion  of  “Decision  Support 
System.”  This  will  require  a  high  level  review  of  300  pages,  with  attention 
paid  only  to  those  meeting  the  previously  stated  criteria.  Each  relevant  Task 
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Decision  Requirement  will  be  reviewed  and  then  checked  for  its  location  in 
the  FAN. 

CMC  also  produced  two  Composite  Mission  Scenarios.  Although  they  have  no 
direct  impact  on  ACWA,  the  Composite  Mission  Scenarios  (30  pages)  will  be 
read  for  ideas  for  the  visualization  storyboard  prototype. 
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D  Review  of  Lockheed  Martin’s  Blueprint/Functional 
Model 

D.1  Overview  of  the  Blueprint/Functional  Model  Framework 

Lockheed  Martin  conducted  a  Blueprint/Functional  Model  for  the  Combat  System  Functional 
Requirements  of  Canadian  Warships.  The  Blueprint  Model  is  a  data-flow  model  through  a  10 
level  decomposition  of  operational  requirements.  Figure  4  depicts  the  top  level  of  the 
decomposition  hierarchy.  Lockheed  Martin  refers  to  each  box  as  a  Function  -  which  is  an 
operational  requirement  at  any  level  of  decomposition.  Figure  5  is  the  data  (which  LM  calls 
Inter-Function  Messages )  input/output  diagram  for  node  1.2  (shown  in  figure  4).  A  more 
-  detailed  definition  of  these  artifacts  can  be  seen  in  Appendix  B. 


Figure  4:  LM's  Blueprint  Functional  Model  Hierarchy,  Level  1 


Figure  5:  LM's  Blueprint/Functional  Model  Input/Output  Diagram,  Level  1 
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D.2  Places  of  Potential  Input  from  Lockheed  Martin’s 
Blueprint/Functional  Model  to  ACWA 

The  following  diagram  shows  the  potential  places  that  LM's  Model  may  provide  input  to 
ACWA.  None  of  LM’s  artifacts  are  similar  to  those  of  ACWA.  However,  Functions  and  Inter- 
Function  Messages  have  potential  to  provide  insights  to  ACWA. 


10/14/2004 
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Figure  6:  Lockheed  Martin's  Blueprint/Functional  Model  and  ACWA  Relationship 
Diagram 
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D.3  Summary  of  Insights  from  Blueprint/Functional  Model 


The  insights  provided  to  ACWA  by  Lockheed  Martin's  Blueprint/Functional  model  will  be 
limited.  The  Blueprint/Functional  model  resembles  a  data  flow  model,  based  on  system- 
requirements  for  Canadian  Warships  identified  by  Lockheed  Martin.  The  mapping  of  artifacts  in 
LM’s  modeling  language  to  a  goal-means  decision  centered  representation  will  require  a 
considerable  amount  of  transformation.  Lockheed  Martin’s  artifacts  will  be  considered  as  a 
last  resort  to  help  All  gaps  in  cognitive  work  requirements  or  information  requirements 
that  may  be  identified  in  the  ACWA  model: 

The  Inter-Function  Messages  -  the  data/direction/control  and  execution/results 
(inputs  and  outputs)  of  the  de-aggregated  operational  requirements,  although 
mostly  data-centric,  may  provide  insights  to  data  in  context  for  information 
requirements  in  ACWA.  The  list  of  Inter-Function  Messages  is  179  pages 
long. 

The  Functions  identified  by  Lockheed  Martin  may  give  insights  to  Cognitive 
Work.  The  list  of  Functions  is  86  pages  long. 

Although  a  search  may  be  specifically  CWRs  or  IRRs,  additional  review  of  other 
sections  in  the  report  may  need  to  be  done  in  order  to  correctly  translate  the  LM 
analysis  for  ACWA  modeling.  The  total  length  of  LM’s  report  is  1956  pages. 
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E  Review  of  HumanSystems  Inc.’s  Functional  Analysis 


E.1  Overview  of  HumanSystems  Inc.’s  Functional  Analysis 

Human5_y.s/e7HA’  Inc.  conducted  an  analysis  of  the  Canadian  Naval  Task  Group  Operational 
Planning  Process.  The  analysis  of  HumanSv-stew.v  is  similar  to  that  of  CMC’s.  The  major 
difference  is  that  CMC  focused  on  the  roles  or  missions  of  an  actor  and  HumanS  vsfem.y  focused 
on  the  missions  of  a  process,  regardless  of  who  (or  what  -  including  computer  systems)  is 
involved  in  performing  the  process.  Figure  7  illustrates  the  block-diagram  style  de-aggregation 
hierarchy  developed  by  HSI.  Another  difference  from  CMC’s  analysis  is  that  HSI  does  not 
designate  the  number  of  levels  of  de-aggregation.  The  end  (stopping  point)  is  once  the  mission 
has  reached  a  “task”  level.  HSI  then  performs  a  Tabular  Task  Analysis  for  EVERY  “block”  in 
the  hierarchy  (not  just  the  lowest  level).  An  example  row  of  this  task  analysis  is  depicted  in 
Figure  8.  A  complete  list  and  description  of  HSI  Functional  Analysis  artifacts  can  be  seen  in 
Appendix  C. 
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Figure  7:  Example  -  HSI’s  De-aggregation  Hierarchy  Diagram 
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Figure  8:  Example  -  HSl's  Tabular  Task  Analysis 
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E.2  Places  of  Potential  Input  from  HSI's  Functional  Analysis 
to  ACWA 

The  diagram  below  shows  the  potential  places  that  HSI's  Functional  Analysis  of  the  Task  Group 
Operational  Planning  Process  my  provide  input  to  ACWA.  Dotted  arrows  depict  where  insights 
may  be  found  for  ACWA.  The  closest  matching  artifact  from  HSI  is  the  Information 
Requirements  -  which  match  ACWA’s  Data.  Most  insights  are  expected  to  come  from  the 
Novice/Expert  Issues  of  the  Tabular  Task  Analysis. 


10/15/2004 


Places  of  Potential  Input  from  HSI’s 
Functional  Analysis  to  ACWA  Diagram 

ACWA  HumanSystem’s 
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Figure  9:  HSI's  Functional  Analysis  and  ACWA  Relationship  Diagram 
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E.3  Summary  of  Insights  from  HSI’s  Functional  Analysis 
into  the  AWW  C2  Project 


The  analysis  produced  by  HSI  is  based  on  a  very  similar  methodology  use  in  the  Mission 
Function  Task  Analysis  by  CMC  Electronics  Inc.  (as  described  earlier  in  section  C  of  this 
report).  Naturally,  the  potential  places  for  input  into  ACWA  are  also  very  similar.  The 

Function  Flow  Model  will  be  used  sparingly  (as  there  is  little  explanation  or  context  for  the 
Functions  identified  in  the  diagram)  as  a  resource  to  fill  gaps  in  the  ACWA  modeling.  The 

following  are  inputs  unique  to  HSI's  Analysis: 


HSI  discusses  the  importance  of  “global”  view  of  the  situation  -  incorporating 
effects  that  ownship  actions/non-actions  have  on  other  ships  within  the  task 
group.  These  types  of  insights  will  be  useful  for  discovering  information 
requirements  in  ACWA.  HSI  discovered  these  insights  during  the  comparison 
of  the  OPP  Functions  to  the  ORO’s  Mental  Model.  The  comparison  of  OPP 
to  the  ORO  Mental  Model  will  be  used  as  a  reference  when  gaps  in 
ACWA’s  “Information  Representation  Requirements”  are  found.  This 
section  of  the  report  is  a  5  page  appended  table. 

HSI's  Task  Analysis  states  *key*  differences  in  the  views  of  an  expert  and  a 
novice  while  performing  a  task.  These  statements  are  expected  to  be  useful  for 
ACWA  in  discovering  cognitive  work  and  information  requirements.  The 
statements  may  provide  insights  directly  for  the  analysis  or  they  may  provide  a 
starting  point  to  ask  for  further  SME  input.  All  Expert/Novice  Issues  will  be 
considered  and  when  appropriate,  addressed  in  the  ACWA  modeling.  This 
will  require  review  of  the  18  pages  of  the  Task  Analysis,  however,  only  a 
fraction  of  the  Tasks  have  an  Expert/Novice  Issue. 

Action  Requirements  and  Information  Requirements  will  also  be  used  as  a 
reference  when  gaps  in  ACWA’s  Cognitive  Work  Requirements  and 
Information  Relationship  Requirements  are  found.  Again,  this  requires  a 
review  of  18  pages  of  Task  Analysis 


25  January  2006 


Page20 


SMA  -  A  ManTech  Group 


Above  Water  Warfare  Command  and  Control  Visual  Decision  Support  Storyboard  Prototype 
W7701-4-2124/001/QCA 


F  Review  of  CMC’s  Information  Flow  and  Processing 
Analysis  Report 

F.1  Overview  of  CMC’s  Information  Flow  and  Processing 
Analysis  Report 

The  Information  Flow  and  Process  Analysis  Report  by  CMC  is  the  third  in  an  increasing  effort  to 
analyze  the  SWC,  ASWC,  and  ORO  positions  of  the  HALIFAX  class  frigate’s  Operations  Room 
by  CMC.  This  analysis  uses  the  results  of  the  Mission  Function  Task  Analysis  Report  by  CMC 
discussed  earlier  in  section  C  of  this  document.  In  the  Information  Flow  Processing  Analysis 
“Subjective  reasoning”  was  used  to  identify  “Task  Sequences”  (which  correspond  to  the 
MFTA’s  First  Level  Functions )  that  are  most  likely  to  “lead  to  the  identification  of  decision¬ 
making  and  information  flow  issues  and  requirements  for  the  ops  room.”  Each  “Task  Sequence” 
was  then  analyzed  and  represented  as  an  Operational  Sequence  Diagram  which  graphically 
depicts  the  information  flow  within  the  “Mission  Segments”  (Task  Sequences)  against  time.  The 
characteristics  of  the  information  in  these  diagrams  include: 

Sources  of  information 

Communication  during  the  task 

Information  processing 

Output  information 


The  symbols  used  to  depict  this  information  flow  are  shown  in  Figure  10  below. 
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■  Discussion  Example 


Figure  10:  Operational  Sequence  Diagram  Symbols 


Fifteen  “Mission  Segments”  or  “Task  Sequences”  were  analyzed.  In  addition  to  the  Operational 
Sequence  Diagrams  (OSDs),  CMC  provided  a  2-page  table  for  each  “Task  Sequence”  which  lists 
a  number  of  information  flow  characteristics,  including  those  listed  above.  An  example  of  this 
table  can  be  seen  in  Figures  1 1  and  12. 
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CSGC 


ANNEX  C  -  INFORMATION  FLOW  AND  OPERATIONAL  SEQUENCE  ANALYSIS  -  Part  1 

Critical  Task  Sequence:  Monitor  and  Maintain  the  RMP/RAP  Top  Level:  1  Conduct  Transit 

First  Level:  1.2  Conduct  Continuous  Surveillance 

Description:  At  sea  continuous  surveillance  is  conducted  to  build  and  maintain  the  Recognized  Maritime  Picture  (RMP)  aud  ihe  Recognized  Air  Picture  (RAP).  Tire  scenario  places  the  Task 
Group  57NM  north  of  LAKOS  Island  on  a  southerly  course.  His  a  multi -threat  environment  The  Operations  Team  must  continuously  gather,  display,  interpret  and  disseminate 
information.  The  information  may  be  gained  by  organic  means  (radar.  EW  and  Conans  intercept)  and  external  such  as  link  or  intelligence  reports.  This  critical  task  sequence  is 
essential  for  situational  awareness  during  the  conduct  of  all  operations. 


Summary  of  Information  Requirements: 

1 .  Details  of  on-going  and  or  planned  activities  including 
specifics  of  tactical  situation  (e.g.  threat/fiiendly  forces,  ROE, 
weather  and  sea  state,  oceanographic  information,  etc. 

2.  Status  of  weapons,  munitions,  systems,  and  related  sensors. 

3.  Mission  requirements  and  objectives. 


Information  Processing  Required: 
1.  Cognitive  (including  memory) 


2.  Discussion 


3.  Intuitive  decisions 


4.  Data  entry 


Potential  Soorcefs)  of  Information: 

1.  Hard  copy  sources  in  fonn  of  SOPs/referenceVdocumenis. 

2.  GCCS/CCS/SSD  -  including  any  linked  information. 

3.  Organic  sensors  and  other  operators. 


Decisions  Required: 

1.  Determine  threat  type  and  level  based  on  the  interpretation  of  all 
available  intelligence  information. 

2.  Determine  validity  of  information  displayed  on  SSD  and  received 
by  other  means. 

3 .  Detamine  the  classification  of  a  contact  based  cq  several  criteria. 

4.  Determine  when  situations  require  the  attention  of  Command  or 
other  team  members. 


Information  Retrieval  Method(s): 

1 .  Visual  -  graphical  and  alphanumeric. 


2.  Auditory 

3.  Memory 


Decision  Making  Processes: 

1.  Cognitive  processing 
(mteipreb'anatyze'assesVdeteimme). 

2.  Operator  assist  tools  -  ID  criteria  check  list. 


Participants: 

CO,  ORO.  OOW,  SWC,  ASWC,  SAC,  ORS,  TS,  EWS. 
AERO,  FC 


Information  Produced: 

Information  Destination^): 

Information  Transfer  Method(s): 

Information  Management  Processes: 

1.  Classification  of  contact 

I .  Internal  -  Command  anti  team 

1 .  SHINCOM  -  internal  and  external  voice 
circuits 

1. 

Sequential  ordering  -  specific  drill  sequence 

2.  Courses  of  action 

2.  External  -  Task  Group 

2.  Electronically'  -  graphical  and  alphanumeric 
via  CCS  and  link 

2. 

Databases  or  infoimation  systems  - 
CCS^GCCS/COWAN 

3.  Infelligence/threat  assessment 

3. 

3.  Auditory -to  ops  room 

3. 

Staieboards 

Figure  11:  Information  Flow  and  Operational  Sequence  Analysis  Part  1 
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ANNEX  C  -  INFORMATION  FLOW  AND  OPERATIONAL  SEQUENCE  ANALYSIS  -  Part  2 


Critical  Task  Sequence:  Monitor  and  Maintain  the  RMP/RAP 

Situational  Awareness  Issues: 

1.  A  better  understanding  of  the  present  tactical  picture  is 
generated  by  the  performance  of  this  sequence  of  tasks. 

2.  Threat  reduction  is  dependant  on  the  operations  room 
team's  ability  to  acquire  and  maintain  an  accurate  picture 
of  the  tactical  situation  and  to  execute  the  warnings 

3.  Threat  assessment  and  course  of  action  determination/ 
recommendation  must  be  associated  with  this  process. 

4.  The  demands  of  intasal  communications  may  create 
situations  where  an  operator  misses  vital  external 
queuing. 


Workload  Issues: 

1 .  Operator  must  concentrate  an  the  tactical  picture  as 
portrayed  on  the  SSD  to  conduct  this  sequence  of  tasks. 


Top  Level: 

First  Level: 

Consequences  of  Error: 

1.  Mission  -  tactical  and  operational.  Improper  prioritization  will 
reduce  the  situational  awareness  of  the  team. 


1  Conduct  Transit 

1.2  Conduct  Continuous  Surveillance 
Timing  Issues: 

1 .  Distance,  speed  and  type  of  contact  dependent 


2 .  Safety  -  own  forces  or  neutral  aircraft  may  be  placed  in  tewms  way 
unnecessarily  by  the  results  of  poor  resolve  procedures. 


2.  The  decision  making  process  must  be  shortened  when 
multiple  contacts  require  analysis. 


3.  The  volume  of  information  that  is  required  by  numerous 
operators  increases  the  time  required  in  the  transmission 
and  receipt  of  information  thereby  reducing  the  time 
available  for  processing  of  the  information. 

4.  Current  situation  -  the  contact  density,  the  number  of 
contacts  within  the  operating  area  causes  time 
compression. 


2.  Priority  must  be  determined  before  the  classification 
process  begins  for  each  contact.  This  is  a  judgement 
decision  of  the  highest  magnitude. 

3.  The  requirement  for  accurate  information  may  create  the 
need  for  operators  to  repeat  verbal  reports. 

4.  The  multi-source  input  environment  increases  the 
workload  on  the  operators  i.e.  the  requirement  to 
multitask;  read  and  listen  and  report  during  the 

5.  Fatigue  and  boredom  ar  e  factors  in  maintaining  the 
RAP/RMP. 


Summary  of  Findings 


1 .  This  critical  task  sequence  is  essential  for  situational 
awareness  during  the  conduct  of  all  operations. 


2.  Information  is  passed  both  visually  and  aurally,  causing 
cognitive  overload. 


6.  Maintaining  situational  awareness  and  making  tactically 
correct  recommendations  becomes  increasing  more  difficult 
in  a  multi-source  data  environment  such  as  a  CPF  operations 
room 

7.  Priority  must  be  determined  before  the  classification  process 
begins  for  each  contact  This  is  a  judgement  decision  of  the 
highest  magnitude. 


3.  Every'  member  of  the  team  must  maintain  situational  awareness  S.  Fatigue  and  boredom  are  factors  in  maintaining  the 


to  ensure  the  proper  priority  is  given  to  each  contact. 


RAP/RMP. 


4.  Threat  assessment  and  course  of  action  determination/ 
recommendation  must  be  associated  with  this  process. 


5.  hr  a  multi-source  data  fusion  environment  this  sequence 
becomes  more  difficult,  takes  a  greater  amount  of  time,  and 
more  cognitive  ability. 


9.  The  volume  of  information  that  is  required  by  numerous 
operators  increases  the  time  required  in  the  transmission  and 
receipt  of  information  thereby  reducing  the  time  available  for 
processing  of  the  information. 

10.  A  Part  Task  Trainer  could  be  useful  in  developing  an 
operators  abilities  with  respect  to  a  number  of  the  tasks  in 
this  critical  sequence. 


Figure  12:  Information  Flow  and  Operational  Sequence  Analysis  Part  2 
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F.2  Places  of  Potential  Input  from  CMC’s  Information  Flow 
and  Processing  Analysis  Report  to  ACWA 

The  diagram  below  graphically  displays  the  artifacts  of  CMC’s  Information  Flow  and  Processing 
Analysis  that  may  have  insights  to  ACWA.  Again,  the  solid  arrows  represent  artifacts  that  are 
consistently  similar  in  content  to  ACWA  artifacts.  Dashed  arrows  depict  artifacts  that  may 
provide  insights  to  ACWA,  however,  are  not  necessarily  similar  to  an  ACWA  product. 
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Figure  13:  Places  of  Potential  Input  from  CMC’s  Information  Flow  Analysis  to  ACWA 
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F.3  Summary  of  Insights  from  CMC’s  Information  and 
Processing  Analysis  Report  to  the  AWW  C2  Project 

The  Information  Flow  and  Processing  analysis  by  CMC  is  focused  on  information  transformation 
and  propagation  through  time  as  current  task  sequences  are  performed.  The  transformation  or 
exchange  of  information  due  to  current  system  and  organizational  practice  is  not  of  use  to 
ACWA.  However,  the  coupling  of  information  flow  analysis  with  task  sequences  may  add  extra 
context  for  ACWA  to  extract  meaning  about  the  importance  of  the  information,  and  the 
transformation  with  respect  to  goals  of  the  domain. 

The  Decisions  Required  identified  by  CMC  list  decisions  associated 
with  a  given  “Task  Sequence.”  Some  of  these  decisions  are 
organizationally  specific  (Determine  content  and  format  of  records  to  be 
kept)  and  are  not  of  use  to  ACWA.  Others  may  provide  insights  to 
ACWA’s  Cognitive  Work  Requirements.  All  Decisions  Required  will 
be  reviewed  for  their  possible  relationships  to  inherit  goals  of  the 
domain.  This  will  require  a  review  of  one  section  on  each  of  15 
pages  included  in  CMC’s  Information  Flow  Analysis  Table. 

The  Summary  of  Information  Requirements  identified  by  CMC  are 
specific  to  a  task  sequence.  Value  for  ACWA  may  be  found  if  these 
requirements  can  be  associated  with  an  inherit  goal  in  the  domain.  All 
of  CMC’s  Information  Requirements  will  be  reviewed  for  insights  to 
ACWA.  This  will  require  a  review  of  one  section  on  each  of  15 
pages  included  in  CMC’s  Information  Flow  Analysis  Table. 
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G  Review  of  HumanSystems  Inc.’s  Recommended 
Improvements  to  the  HALIFAX  Class  Frigate 

G.1  Overview  of  HumanSysfems  Inc.’s  Recommendation 
Methodology 

Human  Systems  Inc.  reviewed  a  number  of  reports  for  their  insights  to  recommendations  for 
improvements  on  the  HALIFAX  class  Operations  Room  Officer,  Sensor  Weapons  Controller, 
and  Assistant  Sensor  Weapons  Controller  positions.  The  following  reports  were  highlighted  by 
HSI: 

■  Deficiencies  in  Command  and  Control  Support  to  the  Command  Team  in  the  HALIFAX 
Class  (Webb,  R.D.G.,  &  Mclean,  D.N.,  1997) 

■  Literature  Survey  for  Issues  in  Naval  Decision  Support  (Bryant,  F.J.,  &  Webb,  R.D.G., 
1999) 

■  Cognitive  Task  Analysis  of  the  HALIFAX  Class  Operations  room  Officer  (Matthews, 
et.al.,  1999) 

■  CPF  Operations  Room  HCI  Deficiencies  Documented  in  the  Maritime  Operational 
Deficiencies  List  (Kubeck,  1999) 


HSI  then  provided  a  short  (6  page)  summary  of  the  findings  from  the  above  reports.  These 
finding  were  then  used  to  identify: 


1 .  Recommended  Immediate  Improvements  for  the  HALIFAX  Class  -  “improvements 
that  could  probably  be  achieved  almost  immediately  with  the  proper  application  of 
known  guidelines  and  recent  research” 

Ex.  Reduce  Alerts,  Follow  Common  Software  Conventions 


2.  Recommended  Additional  Modules  for  the  HALIFAX  Class  CCS  -  “improvements 
that  would  require  significant  research  and  development” 

Ex.  Add  mission  planning  and  preparation  tools,  Improve  conferencing  capability 


3.  Recommended  Long  Term  Measures  for  the  HALIFAX  Class  CCS  -  “improvements 
that  could  only  be  achieved  with  unlimited  time  and  resources” 

Ex.  Review  ops  room  layout,  Employ  user-centered  design  approaches 
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G.2  Places  of  Potential  Input  from  HSI’s  Recommended 
Improvements  to  the  HALIFAX  Class  Frigate  to  ACWA 

This  report  will  have  no  input  for  ACWA’s  domain  modeling.  It  has  been  based  mostly  on  a 
literature  review  of  similar  reports,  and  includes  recommendations  for  improvement  to  the 
command  and  control  system  of  the  HALIFAX  Class  Frigate.  This  report  does  not  include 
information  about  the  domain’s  goals,  cognitive  work  inherit  in  the  domain,  or  information 
needed  to  perform  the  cognitive  work.  A  figure  has  not  been  included  in  this  section  as  it  has 
been  in  the  previous  sections. 


G.3  Summary  of  Insights  from  HSI's  Recommended 
Improvements  into  the  AWW  C2  Project 

Although  this  report  has  no  direct  impact  on  the  ACWA  modeling  of  the  AWW  C2  domain,  it 
may  present  issues  for  system  and  control  room  design  to  be  considered  in  tasks  2  and  5  of  this 
project.  The  recommendations  by  HSI  will  be  reviewed  and  if  found  to  be  in  scope  for  this 
project,  addressed  in  the  design  of  the  decision  support  system  storyboard  prototype  and 
operations  room  layout  design.  This  will  require  a  thorough  review  of  approximately  30 
pages  of  HSI's  recommendation  report. 
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H  Review  of  HSI's  Assessment  of  the  Impact  of  Multi- 
Sensor  Data  Fusion  on  Command  and  Control 
Operations  in  the  HALIFAX  Class  Frigate  and 
Recommendations  for  Measures  of  Performance  and 
Detailed  Test  Plan. 

H.1  Overview  of  HSI's  Recommendations  for  Measures  of 
Performance  and  Detailed  Test  Plan  for  System 
Assessment 

The  report  by  HSI,  Assessing  the  Impact  of  Multi-Sensor  Data  Fusion  on  Command  and  Control 
Operations  in  the  HALIFAX  Class  Frigate:  Recommendations  for  Measure  of  Performance  and 
Detailed  Test  Plan  was  written  to  provide  a  “comprehensive  Test  and  Evaluation  plan  for 
evaluating  the  impact  of  future  command  decision  and  technologies  (COMDAT)  on  human  and 
operational  performance  in  the  HALIFAX  Class  operations  room”.  This  study  resulted  in  3 
outcomes: 

1 .  Potential  Measure  of  Performance  -  creation  was  guided  by  relevant  sources  and 
related  reports. 

Ex.  Accuracy  in  ignoring  irrelevant  info,  Time  to  detect  change  in  threat  status 

2.  A  Plan  for  Piloting  Data  Collection  -  HSI  suggests  performing  a  simulation  to 
assess  performance  and  conducting  test  trials  based  on  3  categories  of  ORO  tasks 
(Recognizing  Air  Picture;  Maritime  Surface  and  Sub-surface  Pictures;  and 
Recognizing  Maritime  Picture  integrated  with  Wide  Area  Picture).  HSI  also  presents 
considerations  while  Preparing  the  Trial,  Running  the  Trial,  and  Analyzing  and 
Reporting  the  Trial. 

3.  Suggested  Approach  to  Evaluation  -  Step  by  step  procedure  to  successfully  perform 
trial  at  NCOT  (testing)  facility  (ex.  “Refine  measures  and  scenarios”).  Presents 
Issues  Concerning  Data,  Research  Design  Trade-offs  and  Sample  Size 
Considerations,  and  other  similar  simulation  -  data  driven  -  test  concerns. 

H.2  Place  of  Potential  Input  from  HSI's  Measures  of 

Performance  and  Test  Plan  Recommendations  to  ACWA 

Although  the  purpose  of  this  report  is  to  provide  recommended  MOPs  and  test  plan,  the  content 
may  have  some  insights  for  ACWA  modeling.  The  Measures  of  Performance  identified  by  HSI 
actually  are  a  breakdown  into  the  cognitive  activity  required  to  achieve  the  major  tasks 
performed  by  the  Ops  Room  Team.  Although  these  cognitive  activities  are  arranged  by  “task,” 
some  may  provide  insights  to  Cognitive  Work  Requirements  and  Information  Relationship 
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Requirements  for  ACWA  modeling.  For  example,  the  MOP  “Time  to  detect  change  in  threat 
status”  suggests  the  importance  of  noting  *change*  in  threat  status,  versus  only  noting 
*absolute*  threat  status. 

The  suggested  testing  plan  and  data  evaluation  sections  of  the  report  do  not  contain  information 
relevant  to  ACWA  domain  modeling.  Because  Measures  of  Performance  are  the  only  input  to 
ACWA  modeling  from  HSI's  Assessing  the  Impact  ofMSDF  on  C2  Operations  in  the  HALIFAX 
Class  Frigate:  Recommendations  for  MOP s  and  Detailed  Test  Plan  report,  a  diagram  of  this 
input  has  not  been  included. 

H.3  Summary  of  Insights  from  HSI's  Measures  of 

Performance  and  Test  Plans  Recommendations  into  the 
AWW  C2  Project 

The  Measures  of  Performance  created  by  HSI  contain  insights  into  cognitive  work  performed  by 
and  information  requirements  needed  by  the  Ops  Room  Team  while  performing  “major  tasks.” 

Each  MOP  will  be  reviewed  and  possible  insights  will  be  noted.  This  will  require  review  of 
a  4  page  table  of  MOPs.  Those  thought  to  be  relevant  will  require  a  fair  amount  of 
transformation  from  their  placement  in  a  task-based  structure  to  a  goal-means 
representation. 

The  suggestions  for  system  testing  presented  by  HSI  are  very  task  and  situationally  based.  It  is 
expected  that  they  will  provide  little  insight  into  the  Decision-Centered  Testing  task  of  this 
project.  However,  the  Assessing  the  Impact  ofMSDF  on  C2  Operations  in  the  HALIFAX  Class 
Frigate:  Recommendations  for  MOPs  and  Detailed  Test  Plan  report  will  be  reviewed  again 
closer  to  the  testing  phase  of  this  project. 
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I  Review  of  HSI's  Review  of  the  TADMUS  Decision 
Support  System 

1.1  Overview  of  HSI's  Review  of  the  TADMUS  Decision 
Support  System  Methodology 

Matthews,  Webb,  and  Bryant  (1999)  conducted  a  Cognitive  Task  Analysis  of  the  Operations 
Room  Officer  position  of  the  HALIFAX  Class  Frigate.  HSI  used  the  results  of  this  analysis  to 
review  the  TADMUS  Decision  Support  System  on  two  dimensions: 

1.  To  determine  in  the  TADMUS  DSS  provides  the  needed  information  for  the 
ORO  to  accomplish  his/her  goals. 

2.  To  determine  if  the  displays  of  the  TADMUS  DSS  are.  consistent  with  the 
ORO  mental  models  of  his/her  domain. 

Both  the  goals  and  mental  models  of  the  ORO  used  for  the  analysis  were  extracted  from  the  CTA 
of  the  ORO  position. 


1.2  Place  of  Potential  Input  of  HSI’s  Review  of  the  TADMUS 
Decision  Support  System  to  ACWA 

As  a  background  for  the  analysis,  HSI  included  the  results  from  the  CTA  of  the  ORO  position 
from  Matthews  et.  al.  (1999).  This  includes  a  list  of  the  “Goals”  of  the  ORO  position.  The 
“Goals”  identified  by  Matthews  et.  al.  are  specific  to  an  actor  in  the  domain  (the  ORO),  which  is 
inherently  different  than  the  “Goals”  of  the  domain  identified  by  ACWA.  Despite  this 
difference,  the  goals  found  by  Matthews  et.  al.  may  provide  insights  for  cognitive  work 
requirements  in  the  ACWA  domain  model.  In  addition,  notes  on  the  ORO  mental  representation 
of  his/her  tasks  (identified  in  the  CTA  of  the  ORO  position)  have  been  included.  This 
information  was  also  presented  in  the  HSI  Functional  Analysis  of  the  Canadian  Naval  Task 
Group  Operational  Planning  Process  report  discussed  earlier  (section  E).  This  information  may 
provide  input  for  Information  Representation  Requirements  of  ACWA  domain  modeling,  or 
Representation  Design  Requirements  during  ACWA  design. 


1.3  Summary  of  Insights  from  HSI's  Review  of  the  TADMUS 
Decision  Support  System  to  the  AWW  C2  Project 

The  results  of  the  review  of  the  TADMUS  DSS  will  provide  little  insights  for  the  AWW  C2 
Project.  The  review  of  the  TADMUS  DSS  was  based  on  the  results  of  a  CTA  of  the  ORO 
position.  The  CTA  used  for  the  comparison  is  inherently  different  than  the  cognitive  modeling 
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of  ACWA.  This  calls  for  ACWA  to  adopt  its  own  methodology  for  comparing  the  ACWA 
model  to  the  DSS  design.  Despite  this  inherit  difference  in  cognitive  modeling  and  the  lack  of 
insight  from  the  results  of  the  review  of  the  TADMUS  DSS,  the  information  contained  in  the 
CTA  of  the  ORO  position  may  provide  insights  for  ACWA  modeling.  The  goals  identified  for 
the  ORO  position  will  be  reviewed  for  their  relevance  to  ACWA  modeling.  This  will 
require  the  review  of  a  1  page  table,  and  transformation  of  any  relevant  information  from  a 
role-based  representation  to  ACWA’s  goal-means  representation. 
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J  Review  of  Pacific  Science  and  Engineering  Group, 
Inc.’s  Studies  of  U.S.  Navy  Air  Defense  Threat 
Assessment  Report 

J.1  Overview  of  U.S.  Navy  Air  Defense  Threat  Assessment 
Report 

The  Overview  of  U.S.  Navy  Air  Defense  Threat  Assessment  Report  by  Pacific  Science  and 
Engineering  Group  Inc.  is  a  summary  of  3  studies  as  follows: 

1)  Study  of  Cue  and  Threat  Level  Relationship  (questionnaire) 

A  baseline  “Threat  Level”  was  found  (1-5)  for  varying  types  of  contact  classifications, 
weapons,  and  radar  emitters.  Participant  were  then  given  a  number  of  “cues”  (e.g.  speed  of 
contact,  altitude,  distance  from  Ownship,  etc.)  with  a  varying  range  of  values  and  asked  to 
describe  how  the  values  would  change  the  assessed  threat  level  of  the  contact.  Figure  14 
depicts  a  sample  of  the  questionnaire. 
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re  14:  Questionnaire  from  Study  One 
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The  study  asked  questions  in  context  of  2  types  of  environments:  Littoral  and  Open  Waters. 
Conclusions  were  made  about  difference  in  Threat  Assessment  between  the  two  environments. 


2)  Study  of  Relative  Importance  of  Threat  Assessment  Cues  (Computer  Based  Experiment) 
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Participants  were  given  an  ID  of  an  Aircraft  (Assumed  Friend,  Unknown  Evaluated, 

Assumed  Enemy).  Available  were  18  additional  hidden  data  values  about  the  contact 
(available  cues).  Participants  were  asked  to  assess  the  ID  of  the  contact  (if  the  given  ID  was 
correct)  and  asked  to  assess  the  level  of  threat  the  contact  posed.  The  participants  could  then 
reveal  the  desired  data  to  make  the  assessments.  The  results  of  the  study  identified  the  “Most 
often  looked  at  data”  and  the  “soonest  looked  at  data.”  The  study  then  labeled  the  top  6 
“critical  cues”  (Origin,  IFF  Mode,  Intelligence,  Altitude,  Airlane,  ES).  Figure  15  shows  a 
screen  shot  of  the  program  used  for  the  experiment. 


3)  Study  of  Effect  of  Conflicting  Information  on  ID  and  Threat  Decisions  (Computer  Based 
Experiment) 

The  third  study  provided  participants  with  an  already  assessed  ID  of  track,  and  also  a  track 
platform.  Participants  were  then  provided  different  levels  of  conflicting  information  and 
again  asked  to  verify  contact  ID  and  assess  threat  level.  The  study  also  asked  participants  to 
verify  track  platform,  assess  intent,  and  provide  a  confidence  level  in  the  participant’s 
decision.  A  sample  screen  shot  of  the  program  used  for  the  study  if  shown  in  Figure  16. 
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Figure  16:  Screen  Shot  from  Study  3  Computer  Program 


All  data  values  were  revealed  to  the  participants.  Conclusions  correlating  assessed  information 
about  contact  with  conflicting  cues  were  made  (e.g.  Participants  most  likely  to  change  ID  of 
Aircraft  with  conflicting  Origin  and  IFF  Mode). 

J.2  Places  of  Potential  Input  from  U.S.  Navy  Air  Defense 
Threat  Assessment  Report  to  ACWA 

The  conclusions  from  the  above  studies  were  extremely  data  centric.  Many  of  the  Findings  from 
the  report  were  stated  statistically.  ACWA  does  not  believe  that  a  mathematical  relationship  can 
be  found  to  describe  how  data  effects  such  complicated  decisions  as  threat  assessment.  Instead 
ACWA  attempts  to  understand  the  transformation  an  expert  does  to  data  in  his/her  head  in  order 
to  make  decisions.  The  findings  were  focused  “situationally”  in  context  (which  suggests 
different  decision  making  for  different  situations).  This  approach  conflicts  with  ACWA  in  the 
sense  that  ACWA  accounts  for  the  fundamentals  of  decision  making,  which  is  consistent  across 
situations,  as  opposed  to  the  results  of  decision  making,  which  will  likely  vary  across  situations. 
In  addition,  the  first  study  used  a  paper  based  questionnaire  while  studies  2  and  3  used  a 
computer  based  questionnaire.  This  approach  takes  decision  making  out  of  its  natural 
environment,  and  assumes  participants  will  be  able  to  recall  or  identify  which  decisions  he/she 
will  make  under  pressure  while  performing  his/her  job.  A  questionnaire  of  this  type  is 
essentially  a  poll  of  expert  opinion,  out  of  context,  and  does  not  make  any  effort  to  understand 
the  underlying  fundamentals  of  an  expert’s  decision  making  strategy.  ACWA  would  seek  to 
understand  the  context  in  which  an  expert  perceives  the  data  presented. 
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J.3  Summary  of  Insights  U.S.  Navy  Air  Defense  Threat 
Assessment  Report  into  the  AWW  C2  Project 

The  results  of  the  reviewed  report  will  not  provide  any  insights  into  the  AWW  C2  Project. 

However,  the  “data  list”  in  the  document  could  be  a  starting  point  for  ACWA  in  a  Guided 
KE  session  in  order  to  identify  Information  Relationship  Requirements  for  decision 
making. 
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Appendix  A  -  Artifacts  of  MFTA  by  CMC  Electronics  Inc. 


Ar\ 

tifacts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Mission  Analysis 

Reports  organizational 
structure,  specifications 
of  tools  used  in  domain, 
and  characteristics  of 
operational 
environment 

The  Mission  Analysis  produced  by 
CMC  is  a  robust  report  of  the  as-is 
domain  and  contains  information 
typically  seen  in  Knowledge 
Elicitation  documents  used  by 
ACWA 

Can  be  used  as  literature-based  KE 
source  to  possibly  identify  high- 
level  goals  of  domain. 

Top  Level 
Functions 

“Mission  Section”  - 
associated  with  roles 
actor/position  plays  in 
domain 

Conduct  Peacetime 
Operations;  Conduct 
Warfare  Operations; 
Conduct  Transit 
Operations 

Top  Level  Functions  are  meant  to 
be  a  foundation  for  further 
decomposition  analogous  to 

ACWA's  purpose  of  the  FAN. 
However,  ACWA  looks  at  goals  of 
the  domain  as  the  foundation  for 
analysis  and  CMC's  task  analysis 
looks  at  the  missions  of  an  actor  in 
an  organization  as  the  foundation 
of  analysis. 

The  MFTA's  Top  Level  Functions 
define  classes  of  operational  tasks 
for  a  given  actor.  The  insights  this 
CTA  artifact  would  give  to  ACWA 
will  be  seen  more  in  their  de¬ 
aggregation.  These  top  level 
functions  may  define  a  scope 
around  the  intrinsic  goals  of  a 
domain. 
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CSCi 


Ar 

ti facts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

First  Level 
Functions 

Encompasses  the 
"activities  involved  in 
completing  the  higher 
level  functions"  -  act  as 
sub-missions  to 
complete  higher  level 
mission 

Conduct  Continuous 
Surveillance;  Maintain 
Communications 

The  first  level  functions  are  the 
second  layer  of  the  CMC's 
hierarchy  and  decomposes  the 
parent  Top  Level  Function  into  a 
list  of  sub-missions  or  First-Level 
Functions.  This  idea  of  de¬ 
aggregation  is  analogous  to  (not 
the  same  as)  ACWA's  supported- 
supporting  relationships  between 
GPNs.  CMC's  MFTA  builds  a 
hierarchy  decomposing  an  actor's 
mission  to  his/her  individual 
"tasks."  In  contrast,  ACWA  builds 
a  network  of  intrinsic  goals  of  the 
domain  -  finding 
demandi  ng/supporti  ng 
relationships  between  the  goals 
(independent  of  actors).  Though 
the  representations  are  inherently 
different,  they  both  act  as  a 
framework  for  the  separate 
methodologies. 

This  level  in  CMC's  MFTA  may 
provide  insights  to  ACWA's  Goal 
Process  Nodes 
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Ar 

ti facts  of  MFTA  by 

Artifact 

Definition 

Example 

Second  Level 
Functions 

"Measurable 

Performance 
Requirements"  to  fulfill 
sub-missions 

Detect  Contact;  Manage 
Hostile  Track;  ID 
Unknown  Track 
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CMC  Electronics  Inc. 


Compared  to  ACWA 


Insights  provided  to  ACWA 


There  is  not  a  1  to  1  match  between 
CMC's  MFTA  and  ACWA 
artifacts.  This  makes  sense  as 
CMC's  model  is  a  5-level 
decompositional  hierarchy. 

ACWA  does  not  specify  the 
number  of  decomposition  levels. 
Because  of  that,  ACWA  may  find 
that  the  decomposition  (levels  of 
support)  to  achieve  an  abstract 
domain  function  (remember, 

CMC's  MFTA  "function"  is  actor- 
role  specific,  not  domain-goal 
related)  can  be  more  or  less  than 
five.  Therefore,  ACWA  may  see 
the  content  in  CMC's 
decompositional  hierarchy  as  any 
one  of:  Goal  Process  Node  (at  any 
level  of  support  in  the  Functional 
Network),  a  piece  of  a  process 
model,  or  cognitive  work. 
Although,  because  CMC's  mission- 
based  framework  is  inherently 
different  than  ACWA  goal-based 
framework,  a  fair  amount  of 
transformation  will  need  to  occur 
to  represent  CMC's  findings  in 


The  Second  Level  Functions  may 
provide  insights  in  multiple  parts 
of  ACWA,  namely  Goal  Process 
Nodes  (both  Goal-sub  node  or 
Process-sub  node)  and  Cognitive 
Work  Requirements 


ACWA. 
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Ar 

tifacts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Third  Level 
Functions 

"Actions"  of  actors  in 
organization 

SWC  Task,  ASWC  Task, 
ORO  Task 

Each  Second  Level  Function  is 
split  into  3  Third  Level  Functions: 
the  SWC  tasks,  ASWC  tasks,  and 
ORO  tasks.  ACWA  does  not 
model  actors  of  a  domain. 

None 

Fourth  Level 
Functions 

"Any  human  activity 
consisting  of  an 
identifiable  amount  of 
subtasks" 

Select  Radar  to  be  used; 
Receive  secured  report 
from  57mm  gun  crew; 
Monitor  SSD  focus; 

Adjust  console 
brightness 

CMC  defines  this  lowest  level  of 
abstraction  as  the  "task"  level. 

CMC  combines  "actions"  and 
"decisions"  as  "tasks."  ACWA  is 
not  concerned  with  actions  of  an 
actor,  and  not  concerned  with  all 
decisions  for  an  actor,  however,  it 
does  model  the  decisions  inherit  in 
a  domain  (independent  of  the 
organizational  structure  or  as-is 
defined  "process"). 

Some  of  CMC's  defined  "tasks" 
may  give  insights  into  cognitive 
work  inherit  in  the  domain,  for 
example,  selecting  among  choices, 
or  monitoring  system  state.  Other 
of  CMC's  "tasks"  will  be  more 
physical  or  system  specific  than 
ACWA  models  -  for  example 
"Adjust  console  brightness" 
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Ar\ 

t if  acts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Task  Description 

Paragraph  explaining 
"task" 

The  operator,  via  the 
SHINCOM,  recommends 
a  change  in  the  degree  of 
readiness  to  the 
ORO/Command.  This 
recommendation  is  based 
on  the  operator’ s 
detection  of  a  significant 
event  and/or  a  change  in 
the  tactical  situation. 

Descriptions  of  goals,  process 
elements,  and  decisions  are  noted 
in  ACWA's  CSEAR  (Cognitive 
Systems  Engineering  Analysis 
Report)  -  however,  ACWA's 
descriptions  are  presented  more 
abstractly,  where  CMC's  MFTA 
presents  the  task  descriptions  in  as- 
in  terms. 

The  descriptions  of  as-is  processes 
and  tasks  presented  by  the  CTA 
report  may  be  used  by  ACWA  as  a 
knowledge  elicitation  source  to 
extract  the  decisions  made  by  and 
information  used  by  actors  within 
the  domain. 

Task  Initiating 
Requirements 

Cue  to  start  task 

Receipt  of  direction  ; 
start  of  shift;  occurrence 
of  a  significant  event 

ACWA  does  not  attempt  to  define 
a  "decision  initiation  cue,"  rather, 
ACWA  considers  the  cues  for 
decision  making  as  the  state  of  the 
system. 

It  is  suspected  that  the  Task 

Initiating  Requirements  found  by 
CMC's  MFTA  are  more 
organizationally/process  specific, 
and  will  not  provide  ACWA  the 
insights  needed  to  display  the 
proper  cue  as  data  in  context. 

Task  Action 
Requirements 

Physical  actions  done 
by  actor  to  fulfill  task 

Read;  Write; 

View/inspect  appropriate 
system  display 

ACWA  has  no  artifacts  defining 
the  current  actions  done  by  actors 

None 
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Artifacts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Task  Feedback 
Requirements 

Cue  end  of  task  has 
been  achieved 

Verbal 

Acknowledgement; 
Appropriate  information 
is  displayed 

ACWA  does  not  produce  an 
artifact  describing  the 
organizational  cue  that  a  task  has 
been  achieved.  Rather,  the  ACWA 
methodology  provides  feedback 
through  a  continuous  display  of 
system  state  through  graphic 
visualizations  of  data  in  context. 

It  is  suspected  that  the  Task 
Feedback  Requirements  found  by 
CMC's  MFTA  are  more 
organizationally/process  specific, 
and  will  not  provide  ACWA  the 
insights  needed  to  display  the 
proper  cue  as  data  in  context. 

Task  Information 
Requirements 

Data  used  to  perform 
task 

Status  of  Weapons 
Systems/  Munitions; 
Details  of  on-going  and 
or  planned  activities; 
Mission  requirements 
and  objectives 

ACWA  produces  "Information 
Relationship  Requirements"  which 
are  defined  as  the  *data  in  context* 
necessary  for  decision  making. 
CMC's  MFTA's  Task  Information 
Requirements  are  more  data- 
centric. 

The  data  listed  may  provide 
insights,  however,  the  relationship 
or  meaning  needed  in  the  data  to 
support  decision  making  will  need 
to  be  explored. 

Task  Information 
Available 

Current  source  of  data 

Required  information  is 
available  via  the 

CCS/SSD;  Required 
information  is  provided 
by  other  operators 

ACWA  does  not  produce  an 
artifact  defining  the  current  source 
of  data 

The  Task  Information  Available 
will  not  provide  insights  for 

ACWA  cognitive  modeling.  This 
information  may  be  useful  when 
implementing  the  designed 
decision  support  system 

Task 

Communication 

Requirements 

Physical  method  of 
communication 

Voice  (SHINCOM); 

Face  to  Face  (verbal); 

ACWA  does  not  model 
communication  between  actors 

None 
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An 

t if  acts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Task  Decision 
Component 

Rating  and  description 
of  decisional 
requirements  associated 
with  each  task  (Fourth 
Level  Function). 

Measure  used  as  an 
"indication  of  the  level 
of  the  operators  mental 
resources  required  for 
successful  task 
completion"  —  rating  1 
(all  decision)  to  10  (no 
decision) 

This  task  is  a  decisional 
task  consisting  mostly  of 
cognitive  activity  only  a 
single  aspect  is  being 
considered;  This  task 
includes  visual,  auditory, 
and  psychomotor 
components  with  only 
modest  decisional 
aspects  to  any  cognitive 
content. 

ACWA  does  not  attempt  to  "rate" 
the  level  of  cognitive  demands  a 
decision  employs  on  an  actor. 
ACWA  only  models  those  "tasks" 

(to  use  CMC's  MFTA  term)  that 
require  cognitive  work  -  and  as 
ACWA  says,  it  models  decisions. 
CMC's  MFTA  models  additional 
"tasks"  that  may  not  necessarily 
require  cognitive  work. 

It  is  suspected  that  CMC's  MFTA 
"tasks"  that  rate  high  on  the 
"decisional  requirements"  scale 
may  give  insights  of  cognitive 
work  requirements  for  ACWA 

Task  Decision 
Requirements 

Explains  the  cognitive 
work  required  in  the 
task 

The  operator  must  assess 
the  current  situation  and 
decide  on  the  best  course 
of  action  to  recommend 
to  the  ORO/Command  ; 
The  operator  must 
determine  the  best  sensor 
for  the  mission; 

This  CMC  MFTA  artifact 
discusses  the  cognitive  work 
performed  by  an  actor  -  similar  to  a 
discussion  of  "Cognitive  Work 
Requirements"  created  by  ACWA 

The  Task  Decision  Requirements 
will  provide  insights  for  Cognitive 
Work  Requirements 

Task  Frequency 

A  qualitative 
measurement  of  how 
often  a  task  is 
performed 

Rarely;  Occasionally; 
Frequently 

ACWA  does  not  model  frequency 
(of  decisions). 

None 
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Ar 

ti facts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Task  Accuracy 
Required 

A  qualitative 
measurement  of  the 
importance  of  precision 
in  a  given  task 

Low;  Medium;  High 

ACWA  does  not  model  "accuracy 
required" 

None 

Task  Interaction 
Required 

List  of  actors  involved 
in  task 

ORO  -  Command; 
SWC/ASWC  -  ORO 

ACWA  does  not  model  the 
interaction  between  actors  in  a 
domain 

None 

Task  Criticality 
Rating 

A  qualitative  measure 
of  the  impact  of  task 
not  complete,  or 
completed  incorrectly 

Scale  1  through  10 

ACWA  does  not  produce  an 
artifact  analogous  to  Task 

Criticality  Rating 

None 

Adjusted  Task 
Criticality  Rating 

A  qualitative  measure 
of  the  impact  of  task 
not  complete,  or 
completed  incorrectly  - 
adjusted  to  account  for 
human  limitations  and 
as-is  process 
deficiencies 

Scale  1  through  10 

ACWA  does  not  produce  an 
artifact  analogous  to  Adjusted  Task 
Criticality  Rating 

None 
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An 

tifacts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Task  Criticality 
Rationale 

Explains  the  effect 
criticality  has  wrt  the 
criteria  used  to  assess 
Task  Criticality 

This  task  has  high 
potential  to  affect 

Mission  Effectiveness/ 
Completion,  and 
Efficiency  if  improperly 
performed.  The  demands 
of  this  task  to  not 
approach  human 
performance  limits. 

Further  analysis  of  this 
task  may  result  in 
improvement. 

ACWA  does  not  produce  an 
artifact  analogous  to  Task 

Criticality  Rationale 

None 

Task  Completion 
Times 

"Task  performance  wrt 
existing  system" 

Most  Likely,  Max  and 

Min  time  for  each  task 

ACWA  does  not  measure  current 
performance  of  existing  system 

None 

Risk  Mitigation 

Methods  identified  for 
improved  task 
performance 

Provision  of  additional 
training;  Change  of 
procedures; 
Software/Hardware 
solution;  Provide  mission 
planning  tools/  decision 
aids;  Improved  Alerts 

ACWA  does  not  produce  an 
artifact  analogous  to  Risk 

Mitigation. 

CMC's  "tasks"  with  a  risk 
mitigation  of  "Provide  decision 
aids"  -  may  provide  a  focus  area 
for  ACWA  to  consider  in  creating 
a  decision  support  system 

Function 

Allocation 

An  analytical  method  of 
prescribing  a  task  to 
either  the  human  or  the 
machine 

Allocation  :  Human 

Score:  0.073  Rationale: 
Weighted  Sum 

ACWA  does  not  produce  an 
artifact  analogous  to  Function 
Allocation 

None 
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An 

t if  acts  of  MFTA  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Composite 
Mission  Scenario 
(not  included  in 
Figure  3  in 
Section  C) 

An  event  by  event 
walkthrough  of  a 
possible  situation 
encountered  by  the 
crew  on  the  HALIFAX 
class  ship 

Two  scenarios  were 
created:  a  peace  time 
counter  drug  operation, 
and  a  wartime  operation 
involving  multiple 
threats  in  littoral  waters. 

ACWA  does  not  formally  create 
"scenarios" 

The  scenarios  created  by  CMC 
may  give  ideas  for  the  storyboard 
prototype  in  task  2  of  this  project. 

Table  1:  Artifacts  of  CTA  by  CMC  Electronics  Inc. 
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Appendix  B  -  Artifacts  Blueprint/Functional  Model  by  Lockheed  Martin 


Artifacts  of  Blueprint/Functional 

Model  by  Lockheed  Ma 

rtin 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Requirements 

Qualities  or 
capabilities  identified 
through  a  literature 
review  of 
requirements  for 
similar  systems 

The  FCSC  CCS  shall  have  the 
capability  and  the  HCI,  under 
Command  Team  control,  to 
provide  an  audio  and  visual 
warning  to  the  Command 

Team  MFDs  when  the  Engage 
Fire  Sequence  mode  is  being 
initiated  against  an  AAW 
threat;  The  FCSC  CS  shall  be 
able  to  conduct  AAW 
operations  in  a  hostile  EW 
Environment 

ACWA  produces 
"Representational  Design 
Requirements"  at  the  end  of  the 
domain  analysis  to  define  the 
visual  characteristics  of  the 
system  needed  to  support  the 
decisions  identified  in  the 
analysis.  LM's  Requirements 
are  defined  prior  to  domain 
analysis  -  as  the  "starting  point" 
for  the  analysis. 

ACWA  modeling  focuses  on 
supporting  decision  making  in  a 
domain.  LM's  requirements  are 
very  situationally  based,  and 
provide  little  insights  on  the 
cognitive  work  or  decisions 
associated  with  the  situation. 
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Artifacts  of  Blueprint/Functional 

!  Model  by  Lockheed  Ma 

\rtin 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Functions 

Desired  characteristics 
of  the  system  as  means 
of  fulfilling 
"operational 
requirements" 

Determine  Rules  of 
Engagement;  Manage  Transit 
Planning 

ACWA  produces  a  Functional 
Abstraction  Network  of 
supported-supporting 
relationships  of  goals  intrinsic  in 
a  domain.  This  representation  is 
then  used  to  identify  places  of 
cognitive  work/information 
requirements  intrinsic  in  the 
domain.  LM  uses  a  10  level 
hierarchy  to  depict  the  de¬ 
aggregation  of  operational 
requirements.  This 
representation  is  then  used  to 
locate  "Inter-Function 

Messages"  as  data  flow  within 
the  domain. 

Some  of  the  "Functions"  listed  by 
LM  sound  "decision-like"  and 
may  provide  insights  to  cognitive 
work  requirements 
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Artifacts  of  Blueprint/Functional 

Model  by  Lockheed  Ma 

\rtin 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Inter-Function 

Messages 

required 

data/direction/control 
and  execution/results 
of  the  functionality 

aaw_ownship_sensor_calc_cov 
messages,  from  the  Collate 
AWW  Sensor  Coverage 
function,  provides  the 
predicted  FCSC  AWW  sensor 
performance  in  the  current 

AAW  environment  to  the 
Coordinate  Mission  AWW 
Sensor  Coverage  Calculation 
function 

Inter-Function  Messages  define 
data  and  data  sources  used 
within  LM's 

Blueprint/Functional  Model. 
ACWA  produces  information 
requirements  needed  to  perform 
cognitive  work  in  the  domain. 

Most  of  the  Inter-Function 
Messages  are  very  data  centric. 
However,  some  (like  the  given 
example)  may  provide  insights  of 
data  in  context  ("Predicted 
sensor  performance  with  respect 
to  current  environment") 
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Artifacts  of  Functional  Analysis  of  the  Canadian  Naval  Task  Group  Operational  Planning  Process  by 
_ _ _ HumanSystems _ _ 


Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Top  Level  Functions 

System/Mission 

Objectives 

Prepare  for  Watch; 
Conduct  Watch 

Top  Level  Functions  are  meant  to 
be  foundation  for  further 
decomposition  analogous  to 
ACWA's  purpose  of  the  FAN. 
However,  ACWA  looks  at  goals  of 
the  domain  as  the  foundation  for 
analysis  and  HumanSystems' 
Function  Analysis  is  situationally  or 
mission  specific. 

The  Top  Level  Functions  define 
classes  of  operational  tasks.  The 
insights  this  artifact  would  give  to 
ACWA  will  be  seen  more  in  the  de¬ 
aggregation.  These  top  level 
functions  may  define  a  scope 
around  the  intrinsic  goals  of  a 
domain. 

Successive/Intermediate 

Functions 

The  de-aggregate  steps 
from  mission  objective  to 
specific  tasks 

Update  Awareness; 

Check  time  remaining; 
Conduct  Briefing 

This  idea  of  de-aggregation  is 
analogous  to  (not  the  same  as) 
ACWA's  supported-supporting 
relationships  between  GPNs. 
HumanSystem's  Functional 

Analysis  builds  a  mission  to  task 
hierarchy.  ACWA  builds  a 
supported-supporting  network 
among  intrinsic  goals  of  the  domain 
as  a  whole  -  independent  of 
situation. 

These  successive  mission  to  task 
levels  are  situationally  and  “task" 
based,  however,  may  provide 
insights  to  Goal  Process  Nodes, 
Process  Models,  Cognitive  Work 
Requirements;  or  Information 
Relationship  Requirements 
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Artifacts  of  Functional  Analysis  of  the  Canadian  Naval  Task  Group  Operational  Planning  Process  by 
_ _ _ _ _ HumanSystems _ _ 


Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Tasks 

Actions  to  achieve 
mission 

Collect  Data;  Collate 

Data;  Format  Data;  Check 
Report;  Send  Report 

The  Low  Level  Functions  act  as 
tasks  that  need  to  be  completed  by 
either  an  actor  or  a  computer.  The 
focus  of  "system  tasks"  rather  than 
an  "actor's  tasks"  is  similar  to 

ACWA  in  a  sense  that  the  unit 
being  analyzed  is  the  human- 
environment-technology  "system," 
not  a  specific  actor  or  computer 
system. 

Although  ACWA  is  not  mission 
specific  like  HumanSystems' 
Functional  Analysis,  the  "tasks" 
identified  may  provide  insights  to 
Process  Models  or  Cognitive 

Work  inherit  in  the  domain  for 

ACWA  modeling. 

Performed  By 

Actor(s)  allocated  to  task 

Planning  Staff;  Higher 
Command;  Cdr 

ACWA  does  not  identify  actors' 
roles  in  the  domain 

None 

Initiating  Stimulus 

An  event  (usually 
operational)  dictating  the 
start  of  the  task  -  usually 
an  output  or  product 
from  another  task 

Decision;  CONOPS; 
Recommended  COA 

ACWA  sees  initiating  stimuli  as  a 
state  of  the  system,  which  ACWA 
would  model  as  an  information 
relationship  requirement. 

None 

Input 

Organizational  Products 
used  for  task 

List  of  factors;  Cdr's  input; 
CONOPS 

ACWA  does  not  model  the 
organizational  products  of  a  domain 

None 

Information 

Requirements 

Sources  of  information  / 
resources  to  complete 
task;  also  data  used  to 
complete  task 

time,  space,  and  distance; 
morale;  Costs  and  risks; 
CF  Force  Employment 
Manual 

ACWA  models  data  in  context.  The 
data  needed  to  perform  a  task  may 
be  useful,  however,  further  analysis 
will  need  to  be  done  to  identify  the 
information  it  provides. 

Similar  to  what  ACWA  defines  as 

Data 
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Artifacts  of  Functional  Analysis  of  the  Canadian  Naval  Task  Group  Operational  Planning  Process  by 

HumanSystems 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Action  Responses 

sub-tasks  or  procedures 
to  complete  task 

Develop  COAs;  Review 
the  situation;  Staff 

Analysis 

These  as-is  procedures  and  actions 
may  provide  information  as  a  KE 
source  for  ACWA 

May  provide  insights  to  Process 
Models  or  Cognitive  Work 
Requirements 

Output 

Organizational  Products 
produced  by  task 

Key  dates  and  time; 
schedule  of  activities; 
Information  brief 

ACWA  does  not  model  the 
organizational  products  of  a  domain 

None 

Novice/Expert  Issues 

Knowledge  about 
completing  task  that  is 
not  necessarily  apparent 
to  a  non-expert 

Should  be  performed  at 
high  level,  avoid  detail; 
Understanding  of  higher 
command  intent; 
Generating  assumptions 
and  recognizing  own 
assumptions;  Using 
internet  to  acquire  a  great 
deal  of  information 

ACWA  obtains  most  information 
through  Subject  Matter  Experts,  to 
identify  cognitive  work  and 
information  requirements  inherit  in 
the  domain.  These  issues  may 
provide  direct  insights  for  ACWA,  or 
areas  for  further  analysis. 

Expert  comments  about  the 
completion  of  tasks  have  potential  to 
provide  insights  to  Cognitive  Work 
Requirements  to  complete  a  task  or 

Information  Relationship 
Requirements. 

OPP  Functional 
Analysis  comparison  to 
SC-21  Top-Down 
Functional  Analysis 

HSI  compared  the 
Functions  identified  for 
the  Canadian  Naval 

Task  Group's  OPP  with 
the  Top-Down 

Functional  Analysis  of 
the  US  Navy's  SC-21 
ship  (SC-21  Combat 
Information  Center  Top- 
Down  Function  Analysis, 
1997). 

Findings  suggest  that 
correspondence  between 
the  OPP  and  SC-21 
"functions"  are  generally 
at  high  levels  of 
abstraction.  The  OPP 
was  found  to  correspond 
mostly  to  the  "Plan 

Mission  Function"  of  the 
SC-21. 

The  "SC-21  Tasks"  that  the  author 
corresponds  to  the  OPP  Functions 
are  very  process  specific  and  will 
not  provide  insights  to  ACWA 
modeling.  Also,  the  conclusions 
arrived  at  by  HSI  because  of  this 
effort  do  not  give  information  about 
goal-means  relationships  or 
decisions  inherit  in  the  domain. 

None 

25  January  2006 


Page52 


SMA  -  A  ManTech  Group 


Above  Water  Warfare  Command  and  Control  Visual  Decision  Support  Storyboard  Prototype 
W7701  -4-2 1 24/001  /QCA 


cscc 


Artifacts  of  Functional  Analysis  of  the  Canadian  Naval  Task  Group  Operational  Planning  Process  by 

HumanSystems 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to  ACWA 

Comparison  of 
Functions  for  the 
Canadian  Naval  TG  OPP 
and  ORO  Position  of  the 
HALIFX  Class  Frigate 

HSI  attempted  to  find 
similarities  in  the 
functions  identified  for 
the  ORO  Position 
(Matthews  et  al.,  1999) 
and  the  functions 
identified  for  the  OPP 
process. 

The  comparison  was 
done  by  listing  the  ORO 
CTA  Functions,  and 
noting  the  "Directly 

Linked"  and  "Indirectly 
Linked"  OPP  Functions 

ACWA  does  not  model  processes 
or  actor  roles  in  a  domain.  The 
analyses  of  the  process  or  actor's 
role  may  provide  insights  to  ACWA, 
but  the  comparison  of  the  process 
to  the  actor's  role  does  not. 

None 

Correspondence  of 
Canadian  Naval  Task 
Group  OPP  to  OROs' 
Mental  Models 

Compares  "Tasks" 

(HSI's  Functions)  to  the 
mental  representation  of 
the  ORO 

Defines  Level  of 
abstraction  ("Most 
Concrete"  to  "Model"); 
Level  of  Focus  ("zoomed- 
in"  versus  "zoomed-out" 
mental  display);  Domain 
of  Interest  ("Global"  or 
"Warfare  Domain" 

ACWA  sees  these  categorizations 
as  describing  the  need  for  a  micro 
or  macro  (or  micro-macro)  view  of 
the  situation,  in  addition  to 
describing  the  context  in-which  data 
is  needed. 

For  "tasks"  relevant  to  ACWA 
modeling,  the  information  from  this 
comparison  coupled  with  the 
information  from  the  Tabular  Task 
Analysis  may  give  direction  for 
discovering  Information 
Relationship  Requirements  or 
Representation  Design 
Requirements 
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Appendix  D  -  Artifacts  of  Information  Flow  and  Processing  Analysis  by 
CMC  Electronics 


Artifacts  of  Information  Flow  and  Processinq  Analysis  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Summary  of 
Information 
Requirements 

Data  or 

Information  that 
either  drives  the 
task  (Command's 
intention)  or  that 
is  necessary  to 
perform  the  task 
(status  of 
weapons) 

Classifications  and  characteristics  of 
contact;  Status  of  weapons,  munitions, 
systems,  and  related  sensors 

ACWA  identifies  information 
necessary  to  perform  a 
specific  piece  of  cognitive 
work.  The  Information 
Requirements  identified  by 
CMC  are  linked  to  an  entire 
string  of  "tasks"  (a  Task 
Sequence),  where  the  exact 
decision  requiring  the 
information  is  not  identified. 

CMC's  Information  Requirements 
are  similar  to  ACWA's 
Information  Relationship 
Requirements.  For  CMC's 
Information  Requirements  to 
provide  insights  to  ACWA,  the 
decision  associated  with  the 
information  will  need  to  be 
identified  (rather  than  the  "Task 
Sequence"  associated  with  the 
information  -  as  CMC's  analysis 
has  provided) 

Potential 
Sources  of 
Information 

They  physical 
type  of  source  by 
which  the 
information 
requirements  were 
acquired 

Internal  and  External  Communications; 
Hard  copy  sources  in  forms  of 
SOPs/references/documents/intelligence 
summaries 

ACWA  does  not  identify 
current  sources  of  information 

None 
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Artifacts  of  Information  Flow  and  Processin 

g  Analysis  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Information 

Retrieval 

Methods 

The  sense  used  by 
the  actor  to  obtain 
the  information 
requirements 

Visual;  Memory;  Auditory 

ACWA  does  not  identify  the 
sense  an  actor  currently  uses 
to  extract  information  from 
the  world 

None 

Information 

Processing 

Required 

Type  of  cognitive 
work  applied  to 
information 
during  task 
performance 

Interpretation  of  displayed  information; 
Cognitive  (including  memory); 
Discussion;  Application  of  Theory 

CMC  has  identified  types  of 
information  transformations 
occurring  within  an  actor, 
however,  they  have  not 
identified  the  transformation 
itself.  This  artifact  does  not 
identify  the  cognitive  work, 
only  lists  types  that  may 
occur  during  an  entire  task 
sequence.  This  type  is  not 
linked  to  any  particular 
decision.  ACWA  identifies 
types  of  cognitive  work  (Goal 
Monitoring,  Process 
Monitoring,  Process  Control, 
etc.)  -  and  links  these  types  to 
a  specific  piece  of  cognitive 
work.  Also  -  CMC's  types 
are  specific  to  current 
practice,  where  ACWA 
cognitive  work  types  are 
independent  of  the  current 
system  and  current 
procedures 

None 
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Artifacts  of  Information  Flow  and  Processinq  Analysis  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Decisions 

Required 

Cognitive  Work 
associated  with 
Task  Sequence 

Determine  validity  and  accuracy  of 
information  upon  which  tactical  and 
operational  decisions  are  being  made; 
Determine  if  the  Ship’s  movements  are 
best  controlled  from  the  bridge  or  the 

OR 

ACWA  identifies  cognitive 
work  independent  of  as-is 
tasks  and  procedures.  Some 
of  CMC's  Decisions  Required 
are  organizationally  specific 
(Determine  content  and 
format  of  records  to  be  kept)  - 
This  type  of  decision  is  not 
considered  "Inherit  to  the 
Domain"  by  ACWA. 

This  artifact  is  similar  to 

ACWA's  Cognitive  Work 
Requirements.  The  decisions 
identified  by  CMC  will  need  to 
be  linked  to  inherit  goals  of  the 
domain  in  order  to  provide 
insights  to  ACWA. 
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Artifacts  of  Information  Flow  and  Processing 

q  Analysis  by  CMC  Eh 

sctronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Decision 

Making 

Process 

Type  of  cognitive 
work  applied  after 
information 
processing  in 
order  to  reach  a 
conclusion  about 
the  information 

Options  Analysis;  Cognitive  processing 
(interpret,  analyze,  assess,  determine); 
Automated  Systems 

CMC  has  identified  types  of 
cognitive  work  performed  to 
reach  conclusions  after 
information  processing  has 
occurred.  They  do  not  link- 
this  type  to  the  actual 
decision,  rather,  provide  a  list 
of  types  associated  with  a  task 
sequence.  Some  of  these 
types  are  specific  to  current 
practice  (Options  Analysis), 
others  are  extremely  general 
(cognitive  processing). 

ACWA  identifies  types  of 
cognitive  work  (Goal 
Monitoring,  Process 
Monitoring,  etc.)  and  uses 
these  types  to  define  a 
specific  piece  of  cognitive 
work. 

None 

Participants 

List  of  actors 
involved  in  Task 
Sequence 

CO,  ORO,  SWC 

ACWA  does  not  model  actors 
involved  in  current  tasks 

None 
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Artifacts  of  Information  Flow  and  Processin 

g  Analysis  by  CMC  Electronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Information 

produced 

Results  of 
information 
processing  and 
decision  making 
in  task  sequence 

Recommendation  of  best  course  of 
action;  Engageability  of  air  threat; 
Suitability  of  conducting  a  counter  fire; 
Optimal  equipment  state  for  current 
tactical  situation 

To  ACWA,  the  product  of 
decision  making  ( Information 
Produced )  in  theory  should 
be  able  to  be  reverse- 
engineered  to  discover  the 
decision  (a  product  of 
"Optimal  equipment  state  for 
current  tactical  situation" 
should  have  been  found  from 
cognitive  work  of  "Identify 
optimal  equipment  state  for 
current  tactical  situation"). 
Therefore,  ACWA  does 
model  the  information 
produced  as  cognitive  work, 
and  would  find  it  redundant 
to  restate  the  cognitive  work 
as  the  product  of  the  decision. 

The  Information  Produced  by  a 
task  sequence  may  give  insights 
to  Cognitive  Work 

Requirements  if  the  Information 
Produced  can  be  associated  with 
an  inherit  goal  of  the  domain. 

Also,  it  may  give  insights  to 
Information  Relationship 
Requirements  if  it  can  be 
associated  with  another  piece  of 
cognitive  work. 

Information 

Destinations 

Actor  or  group  of 
actors  to  receive 
information  from 
task  sequence 

Interna]  -  command  team;  External  - 
OTC;  External  -  other  surface  units 

ACWA  does  not  model  actors 
involved  in  current  tasks 

None 

Information 

Transfer 

Methods 

Current  tactics  for 

information 

sharing 

SHINCOM  -  internal  circuits; 

SHINCOM  -  external  circuits;  Auditory 
-  discussion 

ACWA  does  not  model 
current  actions  or  tactics 
taken  to  share  information 
between  actors 

None 
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Artifacts  of  Information  Flow  and  Processing  Analysis  by  CMC  Eli 

sctronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Information 

Management 

Processes 

Current  system 
used  for  data 
storage 

Sequential  ordering  -  specific  drill 
sequence;  Stateboards;  Database  or 
Information  System 
CCS/GCCS/COWAN;  Hard  copy 
message  filing  system 

ACWA  does  not  model 
current  system  or  processes 

None 

Situational 

Awareness 

Issues 

Describes 
attention  issues 
for  actors  during 
currently 

performed  process 
sequences 

In  a  multithreat  environment, 
maneuvering  for  one  threat  may  be 
counterproductive  for  another  threat; 
With  the  ORO  monitoring  the  ASWC 
during  this  sequence  the  maintenance  of 
SA  becomes  more  difficult;  The 
maintenance  of  RMP  will  be 
jeopardized  due  to  the  heightened 
importance  of  the  RAP 

ACWA  does  not  model 
current  processes.  Instead, 
important  states  of  the 
environment  requiring 
attention  focus  will  be 
captured  in  the  representation 
design  requirements  of  an 
information  relationship 
requirement.  CMC’s  SA 
issues  are  more  task-oriented, 
instead  of  describing 
important  environment  states 
requiring  attention. 

None 
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Artifacts  of  Information  Flow  and  Processing  Analysis  by  CMC  Electronics  Inc. _ 

Insights  provided  to 


Artifact 


Consequences 
of  Error 


Definition 

Negative  effects 
of  improper 
performance  of 
task  sequence 


_ Example _ 

Mission  -  Improper  performance  of  this 
task  sequence  can  greatly  effect  mission 
success;  Safety  -  improper  use  of 
"Weapons  VETO  Panel"  places  life  and 
equipment  at  risk 


Compared  to  ACWA 

ACWA  does  not  model 
current  task  sequences, 
therefore,  does  not  model 
negative  effects  of  improperly 
performing  task  sequences 


None 


Timing  Issues 


Criticality  of  time 
performing  task 
sequence, 
suggestions  for 
reducing  time  to 
perform  task 
sequence. 


The  entire  sequence  must  be  conducted 
without  delay  to  ensure  the  safety  of  the 
ship;  Improved  display  aids  including  a 
method  of  alerting  the  team  could 
reduce  the  time  required  in  transmission 
of  information;  The  use  of  pull  down 
windows  based  planning  tool  could 
increase  the  efficiency  of  the  operator 


ACWA  does  not  have  a  time 
component 


None 


ACWA 
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A 

rtifacts  of  Information  Flow  and  Processin 

q  Analysis  by  CMC  Eh 

ectronics  Inc. 

Artifact 

Definition 

Example 

Compared  to  ACWA 

Insights  provided  to 
ACWA 

Workload 

Issues 

Identifies 
organization- 
imposed  tasks  in 
addition  to 
operational  tasks 
that  compete  for  a 
given  actors  time 
and  effort 

Team  management  and  individual 
responsibilities  must  be  balanced.  The 
operator  must  brief ,  attend  briefings, 
and  maintain  situational  awareness; 

The  requirement  for  accurate 
information  may  create  need  for  an 
increase  in  time  spent  in  discussion. 

ACWA  does  not  model 
actor's  workload 

None 

Summary  of 
Findings 

Notes  about 
current  issues  in 
the  OR  related  to 
task  sequences, 
some  suggestions 
for  improvement 

The  use  of  centralized  interactive  status 
board  which  provides  visual  queuing  as 
to  equipment  status  would  reduce 
verbal  communications;  The  operator 
must  interpret  all  available  information 
and  determine  threat  type  and  level 
while  performing  all  other  tasks. 

ACWAfocuses  on  the  success 
of  the  domain,  rather  than 
pointing  out  the  deficiencies 
in  current  practices 

None 
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